155 research outputs found
Causally Denoise Word Embeddings Using Half-Sibling Regression
Distributional representations of words, also known as word vectors, have
become crucial for modern natural language processing tasks due to their wide
applications. Recently, a growing body of word vector postprocessing algorithm
has emerged, aiming to render off-the-shelf word vectors even stronger. In line
with these investigations, we introduce a novel word vector postprocessing
scheme under a causal inference framework. Concretely, the postprocessing
pipeline is realized by Half-Sibling Regression (HSR), which allows us to
identify and remove confounding noise contained in word vectors. Compared to
previous work, our proposed method has the advantages of interpretability and
transparency due to its causal inference grounding. Evaluated on a battery of
standard lexical-level evaluation tasks and downstream sentiment analysis
tasks, our method reaches state-of-the-art performance.Comment: Accepted by AAAI 202
A Causal Inference Method for Reducing Gender Bias in Word Embedding Relations
Word embedding has become essential for natural language processing as it
boosts empirical performances of various tasks. However, recent research
discovers that gender bias is incorporated in neural word embeddings, and
downstream tasks that rely on these biased word vectors also produce
gender-biased results. While some word-embedding gender-debiasing methods have
been developed, these methods mainly focus on reducing gender bias associated
with gender direction and fail to reduce the gender bias presented in word
embedding relations. In this paper, we design a causal and simple approach for
mitigating gender bias in word vector relation by utilizing the statistical
dependency between gender-definition word embeddings and gender-biased word
embeddings. Our method attains state-of-the-art results on gender-debiasing
tasks, lexical- and sentence-level evaluation tasks, and downstream coreference
resolution tasks.Comment: Accepted by AAAI 202
- …